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PILOTED SIMULATOR DISPLAY SYSTEM 'EVALUATION - 

EFFECTIVE RESOLUTION AND PILOT PERFORMANCE 

IN THE LANDING APPROACH 

By Wendell D. Chase 

Ames Research Center, NASA 
Moffett Field, California 

SUMMARY 


A study was conducted in two parts to investigate the quality of a visual 
display in a fixed cockpit piloted simulator and ways of measuring pilot - 
vehicle performance. Part I concerned the effective resolution of a typical 
simulator display relative to that for the real world; Part II concerned 
pilots* estimates of range and altitude from the runway threshold and measures 
of his ability to control the vehicle in the approach and landing. A corre- 
lation analysis of the information from part II was used to indicate the 
degree of association between those performance measures. 

The static display characteristics, as measured by the resolution of 
landolt "C” rings, were found to be degraded by as much as a factor of 12 
when compared to the real world. A further loss of resolution by approxi- 
mately one -third of the static resolution occurred with the moving display 
and was influenced by the apparent motion of the airplane » 

> 

Range estimates to the runway threshold were in error by about 10 percent; 
altitude estimates above the runway threshold were in error by about 20 per- 
cent. Error in range estimates decreased with experience while altitude 
estimates remained relatively constant. 


Performance in the landing approach was very similar to that in actual 
flight, and even included a n duck under” maneuver by each pilot. The termi- 
nation of the landing approach was at higher rates of descent, but touchdown 
distance from the runway threshold was about the same as in actual flight. 


A correlation analysis between the various measures of altitude -range 
estimates, and pilot-vehicle landing performance showed the following: (a) 
that the touchdown error depends on the pilot’s ability to judge altitude 
in the landing approach, and (b) the touchdown error is highly correlated 
with the integrated altitude error, and the correlation indicates difficulty 
in estimating the correct altitude to decrease the rate of descent and to 
initiate the flare. However, the absence of motion feedback, ground effect 
dynamic forces, and vestibular and kinesthetic cues may be partigj 
responsible for these errors. 
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Piloted simulators are being used extensively for research related to 
advanced aircraft and spacecraft. An important component of some of these 
simulators is a display of the outside scene for the crew, decause of limi- 
tations in state-of-the-art electronics and optics, these visual display 
systems do not provide a true picture of the real world. Although consid- 
erable progress has been made in recent years in developing display systems 
for research and training (refs. 1, 2), quantitative studies of the visual 
quality of these systems are needed to relate the display characteristics 
with man-system performance. This information could provide a more rational 
basis for defining requirements and specifications for piloted -simulator 
display systems. Although it was recognized that correlates between basic 
characteristics of visual display systems (e.g., resolution and pilot-vehicle 
performance measures) would be difficult to establish because of pilots' 
adaptive capabilities, it was considered desirable to study an available 
display system to provide some baseline information in this area. 

Accordingly, the present studies had the following primary objectives: 

I. To define, experimentally, the effective static resolution of 
a television display relative to that for the real world, and to mea- 
sure the loss of resolution that results from motion of the aircraft 
at landing -approach speeds. 

II. To determine pilots' ability to estimate range and altitude 
with a simulator display system and to land a representative commercial 
jet transport. 

In the first part of this paper, the effective resolution characteristics 
of the simulator display system are presented for several pilots and compared 
with those for the real world. In the second part of the report, variations 
in performance of several pilots for several performance measures, obtained 
in landing approaches, are provided, and some tentative correlations among 
these pilot performance measures are briefly noted. 


NOTATION 


f 

h 

h 

N 

P 

r 


number of favorable ways 

mean altitude, ft 

mean rate of descent, ft/sec 

event of different ways 

probability 

correlation, dimensionless 
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mean touchdown distance from runway threshold, ft 

mean absolute touchdown error from glide -slope runway intersection, ft 

mean resolvable distance of "C" ring, ft 

mean touchdown airspeed, ft/sec 

mean resolvable angle of !, G n ring, min 

mean standard deviation error at discrete altitudes, ft 

standard deviation of altitude estimates, dimensionless 

standard deviation of range estimates, dimensionless 

standard deviation of h, ft 

m 

standard deviation of S R , ft 

m 

standard deviation of S, ft 
standard deviation of V, ft/sec 


mean i*. ^raised altitude error between command and actual flight path, 
ft 


EQUIPMENT AND METHOD 


Description of Apparatus 

The components of the visual simulator are a television camera, runway 
model (scaled at 3 OO feet = 1 foot), projection system, and cab. The televi- 
sion camera mounted ou a five -degrees -of -freedom carriage assembly is shown 
in figure 1. The belt (runway) transporter with one degree of freedom is 
also shown. The television camera is a General Electric 525 scan line, 

30 frames/sec, 2:1 interlace, 4:3 aspect ratio system. The front projection 
system is a Schmidt Projector, with correction plates and a retro-reflective 
screen with a gain of 2.5» Field of view afforded the pilot, located 10 
feet from the screen, was 50° horizontal and 37.5° vertical. An Elgeet 13 mm, 
F2. 5 wide angle lens was used to produce a unity magnification ratio. 

Experimental Procedures 

Three pilots participated, one Ames test pilot, and two engineers with 
military flight experience. Simulator resolution was investigated by measuring 
the pilots' performance as they viewed stationary or moving objects. The 
performance index used was essentially the pilot’s visual acuity for a test 
object observed under a static or dynamic condition. 
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Resolution characteristics (static) * - A special light box and matrix 
plate (fig. 2) was constructed for the pilots to observe the orle tatlon of 
iandolt "C" rings, i.e., left, down, right, or up. This method, which has no 
form discrimination, was designed to determine the relative acuity of the 
observers as a function of the television resolution. The average light 
reading of the matrix box that minimized blooming at the projector was deter- 
mined to be 66.4 foot-lamberts with a contrast ratio of 99*2 percent. The 
complete matrix box was rotated 45° (figs. 1, 2) because of an unequal hori- 
zontal and vertical scan of the television camera. All "C n rings within the 
matrix thus have the same number of picture elements transmitted by the vid- 
icon. Each individual "C" ring was scaled at 6 feet in diameter from which 
the mean resolvable angle can be determined. 

Resolution characteristics (dynamic) . - In order to assess pilot perfor- 
mance, on the basis of information used for visual contact with the runway 
threshold during aircraft landings, it was necessary to produce relative mea- 
surements comparable to observations by human subjects with normal viewing 
conditions. This was accomplished by using a single Iandolt ”0" ring rotated 
from run to run, and located at a comparable real world wheel height above 
the runway threshold. The pilot’s task was to acquire the runway threshold 
visually during a normal landing approach and, when the orientation of the 
ring became discernible, to activate a switch that would record altitude, 
range to threshold, speed, and other pertinent variables. 

Range and altitude estimates .- The pilot’s ability to estimate altitude 
and range, at discrete points along a normal 3° glide slope to the runway 
threshold, was measured by normalized error responses. Altitude varied from 
a minimum of 30 feet to a maximum of 27O feet, while the range varied from a 
minimum of 500 feet to a maximum of 2500 feet. 

The experimental design required a pilot-run matrix of 25 altitude- 
range combinations that were displayed with equal probabilities in seven 
variations according to a Latin square experimental design. 

Landing -approach performance . - Pilot performance was measured in actual 
landing approaches and in a fixed cockpit landing -approach simulator (fig. 3) 
with the Boeing 707 dynamics. The principal dynamics used were those for the 
aircraft longitudinal response including both the phugoid and short period. 
The pilot’s approach task was to establish a stable, well -controlled rate of 
descent to the runway threshold with visual references and to make a termi- 
nation maneuver including a successful flare and touchdown. Initially, the 
pilot’s altitude was 300 feet, flight path, 3°j ground distance to runway 
threshold, 5731 feet; and the approach airspeed, 135 k. Each pilot made a 
total of 45 approaches and landings. 


RESULTS AM) DISCUSSION 


Display Resolution 

(Static): 100 percent correct detection of M C” ring opening . - The 

results of 25 observations per pilot of the static Iandolt C* ring matrix 


through the television system are summarized in table I. Of primary interest 
are the mean resolvable distance (Sr), standard deviation (as); mean resolvable 
angle (|3), and Snellen acuity for each pilot. 


TABLE I.- STATIC LANDOLT "C" RING SUMMARIES 


Parameter 

Pilot A 

Pilot B 

Pilot C 

Sr 

a S 

3 

Snellen acuity 

846.48 ft 
13.67 ft 
4.87 min 
20/13 

869*86 ft 
3*05 ft 
4.74 min 
20/13 

862.82 ft 
5.29 ft 

4.78 

20/20 


The mean resolvable angle (3), subtended at the eye by the "C" ring 
opening, is a measure of the minimum separable acuity which, in this case, is 
4.79 minutes of arc or 287.4 seconds of arc. The nominal value resolved by 
human subjects under normal viewing conditions is 24 seconds of arc (ref. 3). 
Thus, this is a reduction in resolution equivalent to a reduction in acuity 
of 12 to 1. This ratio may be slightly reduced if the visual threshold dis- 
tance of the landolt n C" ring can be determined from the probability of a 
correct answer. It will be recalled that the probability, P, of a correct 
answer may be defined as: 


P = f/N 


where 

f = number of correct ways 
N ~ number of different ways 

Let 

A = a correct answer 
B ss subject detecting ring position 
C = correct answer by guessing only 

Then 

P(C) = l/4 (Since there are four possible positions of the ”0” ring) 

P(c) = 3/4 

P(B) = 1/2 at threshold (50 percent detected correctly) 

P(B) = l/2 at threshold (50 percent not detected correctly) 

Therefore , 


P(A) = P(B) + P(B)P(C) = 1/2 + 1/2 • 1/4 = 5/8 
The probability of a wrong answer: 

P(A) = P(B)P(C) = l/2 • 3/4 = 3/8 
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Therefore, in 25 tries, the expected number of correct responses at the 
subjects sensitivity threshold is; 

(25)P(A) = 25(5/8) = 15-5/8 

Thus, the just resolvable threshold distance can be determined if each pilot 
can correctly identify 1 6 landolt "C" ring positions consistently out of the 
25*. The results show that ^approximately 100 feet can be added to each pilot’s 
mean resolvable distance (Sr) for each respective threshold distance. Conse- 
quently the televisual resolution ratio Indicates a reduction in acuity of 
about 11:1. 

(Dynamic) . - The dynamic flight landing approach to the runway threshold 
required the pilots to observe the orientation of the landolt ”C M ring as it 
was rotated randomly, but with the same order for each pilot, from landing 
approach to landing approach. Table II summarizes the parameters of mean 
range (Sr), mean altitude (E), their respective standard deviations (03, a^), . 
and mean resolvable angle (3) for which acuity is maximum. 

TABLE II.- LANDING -APPROACH ”0" RING SUMMARIES 


Parameter 

Pilot A 

Pilot B 

Pilot C 

S R 

601.47 « 

565.04 ft 

575-86 ft 

a S 

47.68 ft 

54.22 ft 

65.95 ft 

E 

93.08 ft 

76.70 ft 

89.08 ft 

°h 

6.02 ft 

6.40 ft 

6.28 ft 

3 

6.86 min 

7.30 min 

7.16 min 


The dynamic mean resolvable angle (3) for all pilots if 7. 10 minutes of 
arc or converted to minimum separable acuity is 426 seconds of arc. The 
ratio between static and dynamic minimum separable acuity is: 

static = 287-4 = 0 . 67lt6 

(£}). , 426 

' 'dynamic 

This figure is analogous to the pilot’s ability to discriminate visual detail 
of a moving object and is sometimes called dynamic visual acuity (refs. 4, 5)* 
The change in acuity of approximately 1/3 under simulated pilot approach 
dynamic conditions, which could be expected in the real world comparison, 
shows that the loss in resolving power is strongly influenced by the apparent 
motion of the airplane. It is quite possible that this ratio could be further 
impaired for aircraft with higher approach speeds. 

It is interesting to note from figure 4 that pilot A, whose static acuity 
was the worst, had a better dynamic acuity than pilots B and C. Likewise, 
pilot B, whose static acuity was the best among the pilots, had the worst 
dynamic acuity. Pilot C falls between pilots A and B for both static and 
dynamic acuity. The significance is that the pilot’s dynamic and static 
acuity are not the same. This difference may be influenced by each pilot’s 
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training and Ms particular method of controlling the aircraft in the 
landing approach. 


Pilot Performance Measures 

Range and altitude estimation *- The pilot performance in estimating 
range and altitude are represented in figure 5. The standard deviation 
(a e h) and (a er ) of errors is shown as a relative error since the pilot T s 
response to actual altitude and range has been normalized with respect 
to the actual altitude. Each pilot shows an asymptotic level in estimating 
range after at Isnst five sessions. This indicates some learning has taken 
place. Little learning is indicated for altitude estimates except for 
pilot C. Generally , all pilots estimated range twice as well as they esti- 
mated altitude, which may indicate more horizontal visual cues than vertical 
cues. Although the pilots attempted to judge altitude by the intersection 
of the runway with the horizon, their relative altitude estimate errors still 
exceeded their range estimate errors. 

The mean standard deviation (a Q ) in feet at discrete altitudes is shown 
in figure 6 for all pilots. The figure indicates errors in altitude esti- 
mates are around 15 percent and not 20 percent as indicated from the pilot 
performance curves of figure 5, which were the overall responses. 

Landing performance .- The results of 45 landings per pilot are shown 
in table III. These are mean rate of ^descent at touchdown (ft), mean touch- 
down distance from runway threshold (S), mean touchdown velocity (V), and 
each respective standard deviation. - These data were used to show interpilot 
performance variations with actual flight data. 


TABLE III. - LANDING PERFORMANCE SUMMARIES 


Parameter 

— 

Pilot A 

Pilot B 

Pilot C 

are 

ft 

4.51 ft/sec 

3.72 ft/sec 

2.77 ft/sec 


1.75 ft/sec 

1.50 ft/sec 

I.56 ft/sec 

S 

1703.78 ft 

1368.49 ft 

1841.53 ft 

a sd 

236.66 ft 

451.30 ft 

361.51 ft 

V 

109.65 k 

109.32 k 

124.03 k 

a v 

4,32 k 

5.41 k 

3.17 k 




Among measurement criteria for performance is the absolute distance 
error between command glide -slope intersection of the runway and the point 
at which the pilot lands the aircraft. It can be seen from figure 7 that the 
absolute distance errors (S e ) over the span of the trials for pilots A and C 
have a mean error less than 400 feet, which compares favorably with real 
flight landings (ref. 6), however, pilot B has a greater mean error of about 
700 feet. 
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The lines of regression for figure 7 show trends over vide variations. 
Appreciable negative or positive regression line slopes (± correlation) 
Indicate either an Increment or decrement in learning. However, since the 
slopes for all three pilots are small, learning does not appear to be a 
factor In this particular performance measure. 

The integral absolute altitude error is a measure of how well the pilot 
adheres to the command flight path until touchdown, figure 8 indicates a 
mean integrated altitude error of about 200 feet for pilot A, about 300 feet 
for pilot B, and only ah 't 160 feet for pilot C. The larger errors of 
pilots A and B are related to a "duck-under" maneuver (ref. 6) executed upon 
visually acquiring the runway threshold. 

Table III shows that the simulator mean rate of descent is higher 
(2.77 ft/sec - 4.51 ft/sec) than that recorded in real flight WR conditions 
of about 2 ft/sec (ref. 6). The problem associated with higher' rates of 
descent for the simulated landings indicates that the pilots had some diffi- 
culty in estimating altitude prior to starting the flare. Although the 
“ground effect" equations were not included in the simulation, there may be 
other effects which degrade performance such as missing motion feedback, 
vestibular and kinesthetic cues, or limited image resolution (ref. 7). 

Mean distance traversed from the runway threshold during VFR conditions 
is about 1500 feet for real flight (ref. 6), which is comparable with the 
piloted simulator range (subject means) of 1370-1840 ft, Wheel height above 
threshold recommended by the ICAO (international Civil Aviation Organization) 
standard is 40 feet; flight data from 108 landings at Kennedy International 
show a predominate wheel height of 20 feet (ref. 6); trie simulator data show a 
wheel height of about 35 ft. 

Performance correlates . - Correlation is simply the similarity, in 
direction ana degree, of variations in corresponding pairs of observations 
of two variables. The principal problem of simple correlation is that of 
determining the degree of association between these pairs of observations. 

The aggregate of plotted points was consolidated into averages (means) and 
standard deviations in order to investigate more easily the pilots’ scatter 
diagram that might show a trend. The pilot scatter diagram can be expressed 
by an equation of the trend line called "the line of regression," which is a 
minimum- squared -error linear curve fitted to the scatter diagram. The slope 
of this line depends upon the coefficient of correlation (r), -1 5 r < 1. 

It appears that the closer the points lie to a line of regression, the more 
nearly a simple linear equation expresses the association between the vari- 
ables. Thus, a few measured parameters were thought to contribute to prob- 
lems associated with the landing approach and were tested for a correlation 
coefficient close to ±1 (indicating a definite linear relationship between 
the variables). Those with a correlation coefficient close to zero (indi- 
cating practically no linear relationship) were not included in this report. 

Figure 9 shows the relationship of the standard deviation of velocity 
(cr v ) versus the me€’h integral altitude error (Ph e ) which are shown to be 
highly correlated (r = O.983). J 


•* 8 — 





Figure 10 shows the mean absolute touchdown error (|^e|) versus the 
mean Integral altitude error with a high correlation (r « 0*94). 

m 

Figure 11 shows the mean Integral altitude error (J h e ) versus the 
mean touchdown distance from the runway threshold (§) to be highly corrected 
(r « 0.995). This high correlation indicates difficulty in estimating the 
correct altitude to level off the rate of descent prior to and at the start 
of the flare. 

The relationship between the pilots 1 static standard deviation of 
altitude estimates (er e h) and the simulated flight mean touchdown errors ($) 
are shown in figure 12 to be highly correlated (r - 0*914). This may be 
significant in that the mean distance from runway threshold m ay be due 
chiefly to the pilots 1 inability to judge altitude correctly in the landing 
approach. 

« 

Similarly, figure 13 shows the standard deviation of touchdown error 
(e sd ) versus the standard deviation of static altitude estimates (c^) to 
be highly correlated (r = O. 983 ). This shows that the touchdown error is 
closely related to the pilots’ inability to judge altitude in the landing 
approach. 


CONCLUSIONS 


The resolution characteristics of the display system, determined from 
measurements of static acuity, show a considerably degraded resolution 
equivalent to a reduction in acuity of 12 to 1. An buUitlonal loss of 
resolution (dynamic visual acuity) by approximately one -third occurred 
in the landing approach. 

Static and dynamic visual acuity apparently have a negative correlation. 
Pilots have different static acuity but dynamic acuity may be further influ- 
enced by pilot training and methods of controlling the aircraft in the land- 
ing approach. This may account for a negative correlation,* furthermore, it 
shows that the relative motion of the airplane can cause a loss of resolution 
during a landing approach. 

A television-projected simulator display appears to lack the sharpness 
and clarity of actual flight conditions for the landing approach VFR (visual 
flight rules) maneuver. 

The pilot uses visual cues in both the horizontal and vertical planes 
to estimate range and altitude. The pilots of the simulator visually 
estimated altitude with an error of about 20 percent. There was a corre- 
lation between altitude estimates and touchdown errors. Range estimate 
errors were about 10 percent, but there was little correlation between range 
estimates and touchdown errors. 

Real flight touchdown distance (from the threshold) and rate of descent 
during VFR conditions are somewhat less than those recorded from the piloted 
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simulator* However , these errors, which degrade performance in the simulator, 
may he due to the absence of kinematic feedback, vestibular cues, kinesthetic 
cues and image degradation. 

The correlation analysis of the landing- approach performance has shown 
the following: (l) the touchdown error is closely associated with the pilots 1 
inability to judge altitude in the landing approach, (2) the touchdown error 
is also closely related to the Integrated altitude error which indicates the 
pilot's difficulty in estimating the correct altitude for leveling off the 
rate of descent prior to and while initiating the flare. 
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1. - Landing- approach model assembly. 

2. - Light source and landolt "C" ring matrix. 

3 . - Pilot station and projected runway image. 

4. - Pilots r r ' itic and dynamic resolving angle. 

5 . - Pilot performance curves for estimates of range and altitude. 

6 . - Pilots* average altitude error at discrete altitudes. 

7 . - Pilots’ touchdown error versus landing trials. 

8 . - Pilots* integrated altitude error versus landing trials. 

9 . - Pilot correlation between velocity and integrated altitude error. 

10. - Pilot correlation between integrated altitude error and touchdown 

error. 

11. - Pilot correlation between touchdown distance and integrated 

altitude error. 

12. - Pilot correlation between touchdown distance and static altitude 

error estimates. 

13 . - Pilot correlation between standard deviations of static altitude 

error est ima tes and touchdown error. 
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Figure 3 .- Pilot station and projected runway image. 
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PILOTS' TOUCHDOWN ERROR VERSUS LANDING TRIALS 
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PILOT CORRELATION BETWEEN TOUCHDOWN DISTANCE 
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